Getting to grips with "interactivity": helping teachers assess the educational value of CD-ROMs

Many commercial CD-ROMs are now being marketed as suitable for home and school use, increasingly promoted as covering the National Curriculum. However, such promises are failing to be realised because many CD-ROMS are poorly constructed, consisting simply of a mishmash of images, sounds and video that offer little more than light entertainment. The aim of this paper is to provide guidance on assessing the added value of educational CD-ROMs compared with traditional materials by explicating more the notion of interactivity as it pertains to learning. Introduction Many commercial CD-ROMs are now being marketed as suitable for home and school use, increasingly promoted as covering the National Curriculum. There has been much marketing hype and glowing reviews in the popular press about their benefits for education. For example, a typical claim by Holzberg (1994, 57) is: “thanks to the flashy sound and brilliant graphics that make multimedia CD-ROMs so appealing it’s easier than ever before to turn kids onto learning”. However, such promises are failing to be realised and teachers and parents alike are becoming more wary of spending large amounts of money on “whistles and bells”, having been disappointed with the quality and educational value of CD-ROMs already purchased. A central problem is that many CD-ROMS are poorly constructed, consisting simply of a mishmash of images, sounds and video that offer little more than light entertainment. Such a state of affairs does not mean, however, that we should simply abandon CDROMs as an educational resource. Far from it, since the technology itself provides much scope and opportunity for developing innovative and computationally powerful interactive educational material (Rogers and Scaife, 1997). The key is knowing how to put the interactivity potentially provided by the technology to good use. Essentially this involves designing effective activities that are engaging but also enable the learner to understand concepts and to reflect on and integrate different kinds of knowledge. Some CD-ROMs have been more successful than others in achieving these levels of interactivity. This leads us to ask the question, how can teachers and parents be helped in sorting the wheat from the chaff when confronted with an ever increasing catalogue of CD-ROMs? As one teacher put it to us, “You get the title and a brief resume of what’s on the disk but you get no idea of what it’s like to use—you’re shooting in the dark!”. Importantly, even when a copy of the disk has been obtained, there is surprisingly little guidance on how to appraise the learning value of the interactivity it offers. Most instruments available for reviewing and evaluating educational software are concerned primarily with the accuracy and scope of the content, the appropriate age range and group size for its use, the robustness and usability of the software, the availability of paper-based support materials, and basic technical requirements (eg Squires and McDougall, 1994). Whether the interactivity offered by the software offers advantages over traditional materials remains a much neglected issue. The aim of this paper is to provide guidance on assessing the “added value” of educational CD-ROMs compared with traditional materials by making more explicit the notion of interactivity as it pertains to learning. Interactivity is a key aspect of multimedia software, distinguishing it from traditional “passive” media, ie, books, video and radio. Specifically, learners can interact with multimedia in more and varied ways than they can with traditional media. A problem, though, is knowing how to identify the interactivities that support effective learning compared with those that are largely superficial. Currently, teachers and parents have little to go by. Below we describe a study where we asked teachers to assess two representative science CD-ROMs with the goal of making explicit key aspects of interactivity from their perspective. Using the findings in combination with our own conceptual analysis of interactivity, we have developed a method for helping teachers and others evaluate the educational value of interactivity, exemplified by CD-ROM technology. We start by outlining advice currently available to teachers for selecting CD-ROMs and then present an analysis of interactivity in relation to learning. Guidance available to teachers for selecting CD-ROMs Although classroom-oriented reviews can be found in educational software catalogues they are invariably positive because the aim is to market the disks (eg, The Complete Primary Software Catalogue, 1996). More balanced school-oriented reviews sometimes appear in teaching magazines (eg, the Primary Science Review), but the most comprehensive source is the CD-ROM Titles Review produced by the National Council for Educational Technology (1995). This contains reviews of about 400 disks by people with a wide range of primary classroom experience. The review team were asked to consider a range of aspects including (p. 5): • The adequacy of both content and coverage • The appropriateness to the National Curriculum in England • The appropriateness of Reading Age for Key Stage (KS) 1 and KS 2 pupils 322 British Journal of Educational Technology Vol 29 No 4 1998 © British Educational Communications and Technology Agency, 1998. • The quality of interface design, presentation, degree of interactivity, and range of facilities • The quality of support materials (where submitted) • The overall quality of the title The emphasis of the guide, however, is very much on highlighting the volume of information on the CD-ROMs rather than their interactivity value with respect to learning. Reviews are restricted to about 200 words. Within these often only three or four sentences are to do with “the quality of interface design, presentation, degree of interactivity and range of facilities”. Even within these, the potential value of interactivity is only indirectly referred to. Instead, there is more emphasis on search features and the development of information-handling skills. Almost as an afterthought teachers are advised to look out for CD-ROMs which “allow children to add their own voices and text, or grab pictures, sounds and text to put into their own work” (p. 489). As an alternative, teachers themselves can get more involved in reviewing CD-ROMs in relation to their ideas of what kinds of interactivity are important in a classroom context. One example of this collective self-help approach can be found in the “CD-ROM Advisor”, a magazine aimed at the home market which requests readers to send in reviews commenting briefly on likes and dislikes about a disk, and asking them to rate the following factors: • “fun”—how compelling the disk is • ”quality”—the accuracy of the information and the effort put in by the publishers • “multimedia impact”—its use of movies, sound and animation • “overall satisfaction”—would the user recommend the disk? Here again the focus is on potentially superficial aspects of interactivity that bear little or no relation to learning benefits. In both approaches, therefore, teachers are given little guidance about how the forms of interactivity can map onto their teaching goals and can help them improve children’s learning of a subject. So how can we help teachers more effectively assess and become aware of the value of interactivity when selecting CD-ROMs? To begin to help we need to have a better conceptual framework for analysing interactivity. What is “interactivity?” The term “interactivity” has become ubiquitous. All around us we hear about the alleged benefits of interactive TV, interactive multimedia, interactive video, etc. Paul Sargeant, writing in the Guardian Higher Education Supplement recently, described “the four syllables in-ter-act-ive” as “like the jangle of loose change in the marketing man’s highly sensitive ears” (Sargeant, 1997). Typically, the term interactivity is directed at the level of the interface—the user takes some action (eg, clicking a button or dragging an object on screen) and the computer responds to that action (eg, by playing a sound, running an animation or displaying new text). Many CD-ROM reviews reflect a pre-occupation with this level of “reactive” interactivity, for example “there are lots of Getting to grips with interactivity 323 © British Educational Communications and Technology Agency, 1998. opportunities to press things on-screen and see interesting bits of animation” (CD-ROM Titles Review, 1995, 363). However, this level of description overlooks any notion of a goal underlying the exchanges between the computer and the user, and can include activities as aimless as the user hitting keys at random—what Plowman (1996) refers to as “gratuitous interactivity”. A further aspect of the hype about interactivity is the simplistic assumption that it is a panacea for education. When coupled with multimedia technology it is often assumed that the two will allow “every person to discover knowledge in the pattern that fits their paradigm for learning” (Taylor, 1990, 27). But as pointed out by Reeves (1993, 80) the two cannot automatically improve learning “any more than a library in a school can” and “multimedia without the interpretive acts of learners is only a collection of textual, graphical and audio elements”. The critical question is: what is it about interacting with multimedia that has the potential of engendering more effective learning than with other traditional media? To answer this, we need to have a better understanding of the concept of interactivity in relation to the kinds of learning that can be mediated by interacting with different media. This requires us to have a more theoretically-driven level of explanation that focuses explicitly on the kind of cognitive activities (ie, learning, problem-solving and memory tasks) that take place through the physical activities afforded by the interface. Within the educational technology literature there have been several classifications of interaction types in relation to learning. A common one describes learning as taking place on a continuum, from reactive to proactive (eg, Lucas, 1992; Thompson and Jorgensen, 1989; Rhodes and Abzell, 1985). Here the focus is on mapping models of learning (eg, behaviourist, constructivist) to types of learning environments. For example, a reactive model of interactivity is one which has been designed to support learning through drill and practice/reaction and response mode. Conversely, proactive learning is thought to take place through the user being involved actively in the construction of the knowledge. This includes situations where the computer is used as an object to think with, such as providing tools for children to discover ways of measuring time. Other classifications of interactivity and learning have focused more on helping designers support different kinds of learning, through using different kinds of navigation and user control mechanisms. For example, Sims (1997) discusses a range of interactivities, like “update interactivity” and “reflective interactivity”, where he suggests that applications should be designed to present problems to which the learner is required to respond and which are available to other students to look at and compare their answers with. Whilst these taxonomies of interactivity go some way towards clarifying the notion of interactivity in relation to learning, they overlook critical aspects of user engagement which are an integral part of learning (Draper, 1996; Laurillard, 1995; Giardina, 1992). These are the “internal” processes that go on within learners when interacting with different media and other people (ie, teachers, peers). Whether they are observing 324 British Journal of Educational Technology Vol 29 No 4 1998 © British Educational Communications and Technology Agency, 1998. an animation, browsing a book, answering a quiz, having a discussion or constructing a model, there are different kinds of cognitive activities going on. We need to understand interactivity, therefore, not simply in terms of “physical activities at the interface” or in global terms of supporting models of learning but in terms of the cognitive interplay between internal and external representations that arise in the different settings (see Scaife and Rogers, 1996). By this we mean the process by which people integrate representations of the same and different information. This requires analysing how they learn to read and comprehend the significance of the content of different media and how this is assimilated with their current understanding of a domain, at different stages of the cognitive task. For example, reading and abstracting knowledge from a graphical representation (eg, a diagram) requires making connections between different elements of the display in a temporal sequence, using both internal and external representations in concert. In turn, we need to operationalise our theoretical accounts of interactivity in terms of applied guidance which is useful to different audiences (eg, teachers, parents and designers) for the purpose of designing and evaluating interactivity with their differing goals in mind. We have already discussed elsewhere how to do this as practical guidance for designers (see Scaife and Rogers, 1996; Rogers and Scaife, 1997). There we presented a framework that identified a range of cognitive properties that can be attributed to external representations which make them more or less easy to interact with (see Table 1). We have also described the kinds of interactivity that would be useful for helping university students learn about interface design through using CD-ROMs (see Rogers and Aldrich, 1996). These include a range of hands-on activities where the students are provided with opportunities to put theoretical ideas, models and design concepts into practice. In this paper our focus is specifically on developing a framework, consisting of a set of questions and dimensions, that teachers can usefully employ when thinking about the added value of interactivity for use in their teaching. The study: operationalising the concept of interactivity in terms of teaching goals As mentioned earlier, currently available methods for reviewing educational software have not been useful for guiding teachers to think about the benefits accruing from interactivity. A main objective, therefore, was to develop an “assessment” framework geared to teachers, to enable them to evaluate more effectively the educational merit of CD-ROMs with respect to their interactivity (and thus learning) potential. By evaluation in this context we mean: to carry out a study which systematically assesses the pros and cons of a software package (CD-ROM), against a set of pedagogical criteria. A study was carried out initially to identify those aspects of interactivity in CD-ROMs which teachers thought were important for primary school teaching. A panel was set up consisting of a representative set of primary school teachers. They were asked to assess two science-based CD-ROMs, using a questionnaire, and were subsequently brought together in a focus group to discuss the findings. Based on these results, together with our theoretical analysis of cognitive interactivity, a framework was developed which Getting to grips with interactivity 325 © British Educational Communications and Technology Agency, 1998. was intended to be generalisable to educational domains other than science, and applicable to learners of all ages.